Results Table:

Results Table Analysis:

Over the years, some countries have merged or ceased to exist, which is an important fact that should be acknowledged in future data engineering and analysis.

It can be observed that there are 271 unique countries, but today there are only 186 countries.

The missing values represent games that will be played in the future, and this should be considered during data engineering and analysis.

Key Insights from the Pair Plot

• Most matches have low scores for both home and away teams.
• Majority of matches are played on non-neutral grounds, with more score variation compared to neutral matches.
• High scores are rare, with outliers present but not common.
• Neutral ground matches show a wide range of scores for both teams.

Distribution of Match Outcomes:

The plot shows that home wins are the most frequent match outcome, followed by away wins, with draws being the least common.

Number of Matches Over Time:

The plot shows a significant increase in the number of international football matches over time, particularly from the mid-20th century onwards. There is a noticeable peak in the number of matches around the 2010s, followed by a sharp decline during the 2020-2022 period, likely due to the COVID-19 pandemic impacting global sporting events. The trend reflects the growing popularity and globalization of football, with more matches being played in recent decades.

Distribution of Goals Scored:

The plot shows that most matches have low scores, with both home and away teams typically scoring 0 to 5 goals. Home teams score slightly more frequently than away teams. High-scoring games are rare but do occur.

Geographic Distribution of Matches:

import geopandas as gpd: Used for geospatial data manipulation and analysis, including loading world map data and plotting geographic concentrations of matches.

The plot shows that European and South American countries, particularly England, Germany, and Brazil, have hosted the most international football matches. These regions are major hubs for international football activity.

Shootouts Table:

Goalscorers Table:

Top 10 self goals scorrers

Temporal Distribution of Own Goals

Distribution of Own Goals

The histogram above represents the distribution of own goals scored over different minutes in a match. Each bar indicates the frequency of own goals scored within a particular time interval, with the x-axis representing the minutes and the y-axis representing the frequency of own goals.

Key observations:

• The distribution is relatively uniform across most of the match duration, with own goals occurring at various times.
• A noticeable peak occurs towards the end of the match, around the 80th minute, indicating a higher frequency of own goals in the later stages of the game.
• The presence of a kernel density estimate (KDE) curve provides a smooth approximation of the distribution, highlighting trends and patterns more clearly.

This plot suggests that own goals can happen at any time during the match, but there is a tendency for an increased frequency as the match progresses, especially towards the end.

Distribution of Goals by Minute:

The plot shows the distribution of goals scored by minute in football matches, with peaks around the 45th and 90th minutes, indicating higher goal frequencies just before halftime and the end of regular time. There is also a notable increase in goals during extra time.

Top Scorers:

Normalized Top Scorers:

Normalized Top Scorers

The bar plot above represents the top scorers normalized by the number of matches they have played. The x-axis lists the players, while the y-axis shows the average number of goals scored per match.

Key observations:

• Juan Marezzi and Malcolm Macdonald have the highest goals per match, indicating they are the most efficient scorers in terms of goals scored relative to the number of matches played.
• The normalization process provides a fair comparison by accounting for the varying number of matches played by different players.

This plot helps to identify the most effective goal scorers, regardless of the total number of matches played, offering a more accurate reflection of a player's scoring ability.

Goals by Minute and Scoring Team:

• The distribution of goals scored by the home team (green) and the away team (blue) is fairly balanced throughout the match.
• There is a noticeable peak in goals scored around the 45th minute, likely due to efforts to score before halftime.
• Another peak is observed around the 90th minute, indicating a surge in goals as teams push to secure a result before the end of regular time.
• Extra time (beyond the 90th minute) sees fewer goals but still shows activity, reflecting the added urgency in these periods.

Goals by Minute and Match Outcome:

• Goals scored in matches that end in a home win (green) are consistently higher across all time intervals, particularly in the latter stages of the match.
• Draws (light blue) have a relatively steady distribution of goals, with a slight peak around the 45th and 90th minutes.
• Matches that end in an away win (dark blue) also show peaks around the 45th and 90th minutes, but overall, fewer goals are scored compared to home wins.
• The pattern suggests that home teams have a slight advantage in scoring consistently, which contributes to their higher win rate.

Importance of Scoring First Goal:

Scoring the first goal is associated with a higher chance of winning the match.

Matches where the first goal led to a win were more than twice as frequent as those where it did not.

Adding features

Adding:

• Home team won
• Home team win rate
• Away team win rate
• Home team average goals
• Away team average goals

Our own 6 features

Explanation of our new Features

1. Home Team Recent Form

   • Description: This feature represents the recent performance trend of the home team. It is calculated as the rolling average of the last 5 matches' results, where a win is represented by 1 and a loss by 0.
   • Purpose: This feature provides insight into how well the home team has been performing recently, helping to gauge their current form and momentum.

2. Away Team Recent Form

   • Description: Similar to the home team's recent form, this feature reflects the recent performance trend of the away team. It is calculated as the rolling average of the last 5 matches' results, where a win is represented by 1 and a loss by 0.
   • Purpose: This feature provides insight into the away team's current form and momentum, which can be crucial in predicting match outcomes.

3. Home Team Elo Rating

   • Description: The Elo rating system is a method for calculating the relative skill levels of teams. This feature tracks the home team's Elo rating over time, starting with an initial rating (e.g., 1500) and updating after each match based on the outcome.
   • Purpose: This feature dynamically reflects the home team's skill level, considering both historical performance and recent matches. It helps to predict the likelihood of the home team's success in future matches.

4. Away Team Elo Rating

   • Description: Similar to the home team's Elo rating, this feature tracks the away team's Elo rating over time, starting with an initial rating (e.g., 1500) and updating after each match based on the outcome.
   • Purpose: This feature dynamically reflects the away team's skill level, providing a measure of their capability relative to the home team and other opponents.

5. Home Advantage

   • Description: This binary feature indicates whether the match is played at the home team's ground. It is set to 1 if the match is played at the home ground and 0 if the match is played on neutral ground.
   • Purpose: Home advantage is a well-known factor in sports, as playing at home can provide teams with various benefits such as familiar surroundings, support from local fans, and reduced travel fatigue. This feature captures these potential advantages.

6. Tournament Importance

   • Description: This binary feature indicates the importance of the tournament. Major tournaments like the World Cup, UEFA Euro, and Copa America are marked as 1, while other tournaments, such as friendly matches, are marked as 0.
   • Purpose: The importance of the tournament can influence team performance and strategies. Major tournaments typically see teams performing at their best, making this feature relevant for predicting match outcomes.

These features enhance the prediction model by providing valuable context and historical performance metrics without directly revealing the winner, making them suitable for use in Section C.

Feature selection + Encoding

Imputation

First, we will find how many empty cells we have in each feature.

Do we need to impute?

No, as we can see, the only missing values are in the last 58 lines because the games have not occurred up to the date the assignment was given.

We will simply erase those lines of data.

Running of the models:

from tqdm import tqdm: Used to display progress bars for loops, particularly during the training and evaluation of machine learning models.

The best model based on the classification reports is the SVM, as it has the highest precision, recall, and f1-score overall, tied with bagging and adaboost, all achieving an accuracy of 0.75.

Model Performance Analysis

In our project, we aimed to predict whether the home team won in a series of sports matches. The performance of the different machine learning models that we used varied.

1. Support Vector Machines (SVM): The SVM model performed well with an accuracy of 75%. The precision, recall, and f1-score were balanced, making it a promising choice. However, the computational cost should be considered when dealing with larger datasets - as it was the longest to train.

2. K-Nearest Neighbors (KNN): The KNN model, with an accuracy of 67%, might not be optimal for this task given our dataset. The selection of the k value is crucial and should be treated carefully when working with KNN.

3. Decision Tree: With an accuracy of 73%, the Decision Tree model showed a good balance in precision and recall values. However, exploring ensemble models like random forests or boosting could help improve these scores.

4. Random Forest: Scoring an accuracy of 72%, the Random Forest model is a strong contender. Despite its slightly lower accuracy compared to SVM, it remains a good choice, although the computational resources required might be a consideration.

5. Bagging: The Bagging model, with an accuracy of 75%, demonstrated balanced performance metrics. It's prudent to check the variance and bias trade-off in such ensemble models.

6. AdaBoost: Excelling in performance with an accuracy of 75%, the AdaBoost model is a top performer for our task, provided we have adequate computational resources.

In our specific project, SVM, Bagging, and AdaBoost models all perform comparably well, with each achieving an accuracy of 75%. On the contrary, the KNN model didn't fare as well on this dataset and might not be ideal for our kind of data.

Model Feature Importances

Discussion of Importances

Decision Tree Model

The Decision Tree model heavily relies on home_team_elo and away_team_elo. These features dominate the decision-making process, indicating that team strength, as measured by Elo ratings, is crucial in predicting match outcomes. Other features like country and home_team_recent_form have much lower importances. Interestingly, many features such as home_team, tournament, city, home_advantage, and year have zero importance, suggesting that the model does not consider them relevant for predictions.

Random Forest Model

The Random Forest model shows a more distributed feature importance. Key features are away_team_elo and home_team_elo, followed by away_team_win_rate and home_team_win_rate. This model takes into account a broader set of features, indicating a more nuanced approach to prediction. Features like city, year, country, and home_team_recent_form also contribute but to a lesser extent. Home_advantage and tournament_importance are among the least important. The broader spread of feature importances suggests that the Random Forest model benefits from combining multiple weak predictors to enhance overall accuracy.

AdaBoost Model

AdaBoost emphasizes home_team_elo and away_team_elo, with home_team_win_rate and home_team_avg_goals also being significant. Features such as away_team_win_rate and away_team_avg_goals have moderate importance. Several features, including home_team, country, tournament_importance, and year, have zero importance. This model's focus on a few key features aligns with its boosting nature, which iteratively improves weak learners to enhance performance.

Summary and Insights

All models prioritize Elo ratings for home and away teams, indicating their strong predictive power. Performance metrics like win rates and average goals are also significant, especially in Random Forest and AdaBoost models. Location, temporal, and tournament-related features generally have lower importance, suggesting they are less critical for predicting match outcomes.

Explanation for Differences

• Decision Tree: The high reliance on Elo ratings reflects its nature of making decisions based on the most informative splits, favoring features that provide clear distinctions in outcomes.
• Random Forest: By aggregating multiple decision trees, it captures a broader spectrum of feature importances, leveraging various aspects of the data for a more robust prediction.
• AdaBoost: Focuses on refining the model by iteratively correcting mistakes, which is why it emphasizes key predictive features while downplaying less influential ones.

These differences highlight the unique strengths of each model: Decision Trees for their simplicity and clarity, Random Forests for their robustness and generalization, and AdaBoost for its precision and iterative improvement. Understanding these nuances helps in selecting the appropriate model based on the specific requirements of the prediction task.

Post Analysis of Model Results Feature Correlations

Our detailed analysis of feature correlations and model results has uncovered several intriguing and unexpected insights:

1. Strong Correlations:

   • home_team_win_rate and home_team_avg_goals (~0.72), and away_team_win_rate and away_team_avg_goals (~0.73) are strongly correlated. This is expected because teams that score more goals tend to win more games.
   • home_team_elo and home_team_win_rate (~0.74), and away_team_elo and away_team_win_rate (~0.73) suggest that higher Elo ratings generally align with higher win rates. This indicates that the Elo rating system is effective in capturing team performance.

2. Moderate Correlations:

   • home_team_recent_form and home_team_elo (~0.54) suggest that the recent form of home teams consistently aligns with higher Elo ratings, likely reflecting the current performance trends.
   • home_team_recent_form and home_team_win_rate (~0.54) show that recent form is a good predictor of win rate.

3. Unexpected Observations:

   • home_advantage has a slightly positive correlation with home_team_won (~0.057), suggesting a marginal influence on the likelihood of winning, which might not be as significant as traditionally expected.
   • home_team_won has moderate correlations with home_team_win_rate (~0.21) and home_team_avg_goals (~0.16), indicating higher performance metrics lead to more wins, as expected.
   • Negative correlations between home_team_won and away_team_win_rate (~-0.25) and away_team_elo (~-0.32) imply that stronger away teams significantly reduce the likelihood of home team wins, highlighting the impact of strong opposition.

Feature Importance Analysis

The feature importance analysis across different models reveals several key points:

• Decision Tree Model: Relies heavily on home_team_elo and away_team_elo, while many features like country, home_team_recent_form, and others have negligible or zero importance. This suggests that decision trees focus on the most significant predictors, mainly Elo ratings, which could overshadow other potentially useful features.

• Random Forest Model: Highlights away_team_elo and home_team_elo as key features. Features like city, year, and home_advantage have lower importance. The ability of Random Forests to handle multicollinearity ensures that correlated features do not distort the model’s predictions, allowing it to provide more balanced insights.

• AdaBoost Model: Emphasizes home_team_elo and away_team_elo, with moderate importance on performance metrics like win rates and average goals. AdaBoost’s iterative boosting process focuses on the most impactful features, maintaining robustness even with correlated features.

Why Elo is so Strong

Elo ratings are potent predictors because they provide a dynamic and comprehensive measure of a team's strength based on historical performance and head-to-head results. The system's ability to adjust ratings dynamically, reflecting recent form and the quality of opponents, makes Elo ratings a robust indicator of a team's current capability, essential for predicting match outcomes.

Section D

Data prep

Feature Explanations

1. team_name

   • The name of the team.

2. team_win_rate

   • The percentage of matches that the team has won. It is calculated by dividing the number of wins by the total number of matches played by the team.

3. team_avg_goals

   • The average number of goals scored by the team per match. It is calculated by taking the mean of all goals scored by the team in both home and away matches.

4. team_recent_form

   • This feature represents the recent performance trend of the team. It is calculated as the rolling average of the last 5 matches' results, where a win is represented by 1 and a loss by 0. This provides insight into how well the team has been performing recently, helping to gauge their current form and momentum.

5. team_elo

   • The Elo rating of the team, which is a method for calculating the relative skill levels of teams. The rating starts with an initial value (e.g., 1500) and is updated after each match based on the outcome.

6. home_team_advantage_rate

   • The win rate of the team when playing at home. It is calculated by dividing the number of home wins by the total number of home matches played by the team.

7. avg_goals_conceded

   • The average number of goals conceded by the opponent team per match. It is calculated by taking the mean of all goals conceded by the opponent team in both home and away matches.

8. consistency_rate

   • The consistency in the number of goals scored by the team. It is calculated as the standard deviation of the goals scored by the team in both home and away matches.

9. recent_performance

   • Average number of goals scored in games played in recent years (e.g., since 2019).

10. tournament_performance_major

    • The team's win rate in major tournaments. It is calculated by dividing the number of wins in major tournaments by the total number of major tournament matches played by the team.

11. tournament_performance_friendly

    • The team's win rate in friendly matches. It is calculated by dividing the number of wins in friendly matches by the total number of friendly matches played by the team.